Deep learning-based image quality enhancement of three-dimensional anatomy scan images

ABSTRACT

Techniques are described for enhancing the quality of three-dimensional (3D) anatomy scan images using deep learning. According to an embodiment, a system is provided that comprises a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory. The computer executable components comprise a reception component that receives a scan image generated from 3D scan data relative to a first axis of a 3D volume, and an enhancement component that applies an enhancement model to the scan image to generate an enhanced scan image having a higher resolution relative to the scan image. The enhancement model comprises a deep learning neural network model trained on training image pairs respectively comprising a low-resolution scan image and a corresponding high-resolution scan image respectively generated relative to a second axis of the 3D volume.

TECHNICAL FIELD

This application relates to medical image processing and moreparticularly to deep learning-based image quality enhancement ofthree-dimensional (3D) anatomy scan images.

BACKGROUND

Many applications in computed tomography (CT) imaging, magneticresonance imaging (MRI) and other 3D clinical imaging modalities demandexcellent high-contrast spatial resolution. For example, the ability tovisualize small structures is necessary when trying to locate smallpulmonary nodules, visualizing the temporal bone, imaging small calibercoronary artery stents, or assessing bone fractures to determine acourse of treatment.

The spatial resolution of the CT image is mainly determined by the sizeof the detector elements, as defined by the detector spacing anddetector pitch. Although the spatial resolution can be improved byreducing the detector width and inter-detector distance, development ofnew detectors is time-intensive and incurs a high cost. In addition,smaller detector elements lead to an increase of noise. The imageresolution can also be improved by image processing methods, such asimage sharpening by a high-pass filter, and image deblurring by aLaplacian filter and the Richardson-Lucy algorithm. However, these kindsof filters and deblurring methods increase the noise level. Thus,efficient techniques for generating high-resolution scan images in CT(and other 3D clinical imaging modalities) without increasing the noiselevel are needed.

SUMMARY

The following presents a summary to provide a basic understanding of oneor more embodiments of the invention. This summary is not intended toidentify key or critical elements or delineate any scope of thedifferent embodiments or any scope of the claims. Its sole purpose is topresent concepts in a simplified form as a prelude to the more detaileddescription that is presented later. In one or more embodimentsdescribed herein, systems, computer-implemented methods, apparatusand/or computer program products are provided that facilitate enhancingthe resolution of 3D anatomy scan images with minimal change in noisecharacteristics using a deep-learning enhancement network.

According to an embodiment, a system is provided that comprises a memorythat stores computer executable components, and a processor thatexecutes the computer executable components stored in the memory. Thecomputer executable components comprise a reception component thatreceives a scan image generated from 3D anatomy scan data relative to afirst axis of a 3D volume. The computer executable components furthercomprise an enhancement component that applies an enhancement model tothe scan image to generate an enhanced scan image having a higherresolution relative to the scan image. For example, in variousimplementations, the input scan image can include CT scan imagegenerated from CT scan data with standardized CT image processingtechniques. With these implementations, the enhanced scan imagecomprises a sharpened/higher resolution version of the input scan imagethat has the same noise level and inter-tissue contrast level as theinput scan image. The enhancement model comprises a deep learning neuralnetwork model trained on training image pairs respectively comprising alow-resolution scan image and a corresponding high resolution scan imagegenerated relative to a second axis of the 3D volume, wherein the secondaxis and the first axis are different. For example, in some embodiments,the enhancement model can be trained on CT scan images generated alongthe z-axis and then applied to enhance CT scan images generated along adifferent axis (e.g., the axial axis).

These low/high resolution training image pairs can be generated usingvarious techniques. For example, in some implementations, thelow-resolution scan image comprises a thick slice scan image and thecorresponding high-resolution scan image comprises a corresponding thinslice scan image generated using retro-reconstruction. In otherimplementations, the low-resolution scan image and the correspondinghigh-resolution scan image can be generated using focal spot wobbling.Still in other implementations, the low-resolution scan image and thecorresponding high-resolution scan image can be generated via scanningthe same structure using separate scanners, a low-resolution scanner anda high-resolution scanner, respectively.

In one or more embodiments, the system includes a training componentthat employs supervised machine learning to train the enhancement modelto deconvolve tissue features, contrast features, spatial features andpoint spread function features between the training image pairs underone or more defined constraints. For example, the one or more definedconstraints include at least one of, an intensity threshold constraint,a mask constraint, a spatial constraint, or contrast distributionconstraint. The training component can employ one or more loss functionsto preserve the one or more defined constraints. For example, the one ormore loss functions can include, but are not limited to, a mean absoluteerror (MAE) loss function, a percentage loss function, a perceptual lossfunction, an adversarial loss function, and a point spreadcharacteristics constraining loss function.

According to another embodiment, a system is provided that comprises amemory that stores computer executable components, and a processor thatexecutes the computer executable components stored in the memory. Thecomputer executable components comprise a training component that trainsa deep learning network to enhance the quality of first scan imagesgenerated from first 3D anatomy scan data relative to a first axis of a3D volume. The computer executable components further comprise anenhancement component that employs the deep learning network to enhancethe quality of second scan images generated from the first 3D anatomyscan data or second 3D anatomy scan data relative to a second axis ofthe 3D volume. In various embodiments, the training of the deep learningnetwork comprises training the deep learning network to learn one ormore transformations between training image pairs respectivelycomprising a thick scan image and a corresponding thin scan imagegenerated relative to the first axis of the 3D volume, and wherein theone or more transformation comprise a deblurring transformation betweenthe training image pairs under a deblur constraint based one or morepoint spread function (PSF) characteristics associated with the secondaxis.

In some embodiments, elements described in the disclosed systems can beembodied in different forms such as a computer-implemented method, acomputer program product, or another form.

DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of an example, non-limiting systemthat facilitates enhancing the quality of 3D anatomy scan images usingdeep learning in accordance with one or more embodiments of thedisclosed subject matter.

FIG. 2 presents an example framework for training a deep learningnetwork to enhance the quality of 3D anatomy scan images in accordancewith one or more embodiments of the disclosed subject matter.

FIG. 3 presents an example enhancement model architecture in accordancewith one or more embodiments of the disclosed subject matter.

FIG. 4 presents a flow diagram of an example process for employing adeep learning-based enhancement model to enhance the quality of 3Danatomy scan image in accordance with one or more embodiments of thedisclosed subject matter.

FIG. 5 present phantom CT images with and without deep learning-basedenhancement in accordance with one or more embodiments of the disclosedsubject matter.

FIG. 6 provides a graph comparing MTF test results on phantom CT imageswith and without deep learning-based enhancement in accordance with oneor more embodiments of the disclosed subject matter.

FIGS. 7-9 presents clinical MTF CT images before and after applicationof the disclosed enhancement model in accordance with one or moreembodiments of the disclosed subject matter.

FIG. 10 presents a high-level flow diagram of a high-level flow diagramof an example computer-implemented process for enhancing the quality of3D anatomy scan images using deep learning in accordance with one ormore embodiments of the disclosed subject matter.

FIG. 11 presents a high-level flow diagram of a high-level flow diagramof another example computer-implemented process for enhancing thequality of 3D anatomy scan images using deep learning in accordance withone or more embodiments of the disclosed subject matter.

FIG. 12 illustrates a block diagram of an example, non-limitingoperating environment in which one or more embodiments described hereincan be facilitated.

DETAILED DESCRIPTION

The following detailed description is merely illustrative and is notintended to limit embodiments and/or application or uses of embodiments.Furthermore, there is no intention to be bound by any expressed orimplied information presented in the preceding Background section,Summary section or in the Detailed Description section.

The disclosed subject matter is directed to systems,computer-implemented methods, apparatus and/or computer program productsthat facilitate enhancing the quality/resolution of 3D anatomy scanimages with minimal changes in noise characteristics. In variousembodiments, the 3D anatomy scan images include CT scan images,including multi-energy CT images and material images for dual energy CTand spectral CT. However, the disclosed techniques can be applied toother 3D medical imaging modalities, including but not limited to, MRI,positron emission tomography (PET), ultrasound, and the like. Thedisclosed techniques are further anatomy and acquisition protocol (e.g.,contrast/non-contrast) agnostic.

The disclosed techniques employ a deep learning network model thatreceives a reconstructed scan image (e.g., a CT scan image) as input andgenerates an enhanced scan image as output. The enhanced scan image hasincreased sharpness/resolution or modulation transfer function (MTF)relative to the input image while further preserving the noise andinter-tissue contrast characteristics equivalent to that of the inputimage. The MTF is the spatial frequency response of an imaging system ora component. It is the contrast at a given spatial frequency relative tolow frequencies. On the radiogram, objects having different sizes andopacity are displayed with different grey-scale values. MTF isresponsible for converting contrast values of different-sized objects(object contrast) into contrast intensity levels in the image (imagecontrast). For general imaging, the relevant details are in a rangebetween 0 and 2.0 cycles per millimeter (mm), which demands high MTFvalues. In summary, MTF is the capacity of the detector to transfer themodulation of the input signal at a given spatial frequency to itsoutput. MTF is a useful measure of true or effective resolution, sinceit accounts for the amount of blur and contrast over a range of spatialfrequencies.

The deep learning network model, referred to herein as the “enhancementmodel,” comprises a supervised network that learns deblurring betweentraining image pairs consisting of a low-resolution image and acorresponding high-resolution image. The low-resolution image caninclude or correspond to a thick image obtained/simulated from a largerdetector while the high-resolution image can include or correspond tothin image obtained/simulated from a smaller detector. These trainingimages can be realized either through retro-reconstruction from nativethin and corresponding thick detector realization or focal-spotwobbling. The high-resolution image can be a denoised version as well(e.g., depending on the noise level of the input image, a denoising canprecede the enhancement model).

In this regard, the enhancement model learns deblurring from trainingimages generated only in the scan direction where a thin detector and athick detector configuration equivalent image can be generated, eitherbecause of native resolution or through focal-spot wobbling. A CTdetector (and other modality 3D medical imaging system detectors) willhave similar point spread function (PSF) characteristics in alldirections. This property is exploited to use the trained network indeblurring new scan images generated along other scan directionsrelative to the direction of the training images, as long as the deblurfactor of the network training direction is less than PSF of the otherscan directions. In other words, the disclosed techniques use theseparable nature of the enhancement model transform and PSF similarityto apply the enhancement model transform in any directions of choice. Inone example implementation, the enhancement model was trained on CTimages generated along the z-direction and applied to transform new CTimages generated along the axial direction. However, the proposed methodenables generation of scan images with a sharper resolution/MTF than theimaging scanner system PSF can produce in both the axial z-direction (orin 3D), with or without changing the grid size. The enhancement modeltransform can be used to improve quality of scan images obtained fromlow-resolution scanners, and more importantly to generateultra-high-resolution images from scan images obtained from the highestresolution scanner.

The proposed method also enables training the enhancement model to learnone or more non-linear transformations between input and output imagepairs. For example, the non-linear transformations can be based ontissue specific transforms like denoising, artifact correction orHounsfield unit (HU) dependent smoothing/sharpening. The enhancementmodel can also learn to deconvolve tissue/contrast/spatial-dependent PSFfeatures, with user-defined constraints. The user defined constraintscan be in form of intensity thresholds, masks or spatial information orcontrast distribution. This can also include PSF characteristics foreach of these components (spatial/tissue/contrast).

The types of medical images processed/analyzed using the techniquesdescribed herein can include images captured using various types ofimage capture modalities. For example, the medical images can include(but are not limited to): radiation therapy (RT) images, X-ray (XR)images, digital radiography (DX) X-ray images, X-ray angiography (XA)images, panoramic X-ray (PX) images, computerized tomography (CT)images, mammography (MG) images (including a tomosynthesis device), amagnetic resonance imaging (MR) images, ultrasound (US) images, colorflow doppler (CD) images, position emission tomography (PET) images,single-photon emissions computed tomography (SPECT) images, nuclearmedicine (NM) images, and the like. The medical images can also includesynthetic versions of native medical images such as synthetic X-ray(SXR) images, modified or enhanced versions of native medical images,augmented versions of native medical images, and the like generatedusing one or more image processing techniques.

A “capture modality” as used herein refers to the specific technicalmode in which an image or image data is captured using one or moremachines or devices. In this regard, as applied to medical imaging,different capture modalities can include but are not limited to: a 2Dcapture modality, a 3D capture modality, an RT capture modality, a XRcapture modality, a DX capture modality, a XA capture modality, a PXcapture modality a CT, a MG capture modality, a MR capture modality, aUS capture modality, a CD capture modality, a PET capture modality, aSPECT capture modality, a NM capture modality, and the like.

As used herein, a “3D image” refers to digital image data representingan object, space, scene, and the like in three dimensions, which may ormay not be displayed on an interface. 3D images described herein caninclude data representing positions, geometric shapes, curved surfaces,and the like. In an aspect, a computing device, such as a graphicprocessing unit (GPU) can generate a 3D image based on the data,performable/viewable content in three dimensions. For example, a 3Dimage can include a collection of points represented by 3D coordinates,such as points in a 3D Euclidean space (e.g., a point cloud). Thecollection of points can be associated with each other (e.g., connected)by geometric entities. For example, a mesh comprising a series oftriangles, lines, curved surfaces (e.g. non-uniform rational basissplines (“NURBS”)), quads, n-grams, or other geometric shapes canconnect the collection of points. In an aspect, portions of the mesh caninclude image data describing texture, color, intensity, and the like.

A 3D anatomy image refers to a 3D or volumetric representation of ananatomical region of a patient. In some implementations, a 3D anatomyimage can be captured in 3D directly by the acquisition device andprotocol. In other implementations, a 3D anatomy image can comprise agenerated image that was generated from one-dimensional (1D)two-dimensional (2D) and/or 3D sensory and/or image data captured of theanatomical region of the patient. Some example 3D medical images include3D volume images generated from CT scan data, and MRI scan data. It isnoted that the terms “3D image,” “3D volume image,” “volume image,” “3Dmodel,” “3D object,”, “3D reconstruction,” “3D representation,” “3Drendering,” and the like are employed interchangeably throughout, unlesscontext warrants particular distinctions among the terms. It should beappreciated that such terms can refer to data representing an object, ananatomical region of the body, a space, a scene, and the like in threedimensions, which may or may not be displayed on an interface. The terms“3D data,” and “3D image data” can refer to a 3D image itself, datautilized to generate a 3D image, data describing a 3D image, datadescribing perspectives or points of view of a 3D image, capture data(e.g., sensory data, images, etc.), meta-data associated with a 3Dimage, and the like. It is noted that the term “2D image” as used hereincan refer to data representing an object, an anatomical region of thebody, a space, a scene, and the like in two dimensions, which may or maynot be displayed on an interface.

The term “3D anatomy scan data” is used herein to refer to thecollection of scan data acquired/generated in association with aperformance of a 3D medical imaging scan, such as a CT scan, an MRIscan, a PET scan or the like. For example, 3D anatomy scan data caninclude 1D, 2D and 3D data that can be used to generate a 3D volumetricimage of the scanned anatomy and to generate 2D scan imagescorresponding to slices of the 3D volumetric image from variousperspective/orientations (e.g., relative to the axial plane, the coronalplane, the sagittal plane and other reformatted views). The term “scanslice,” “image slice,” “scan image,” and the like are used hereininterchangeably to refer to a reconstructed 2D image generated from 3Danatomy scan data that corresponds to a computer-generatedcross-sectional image of an anatomical region of a patient.

The terms “thick” and “thin” as applied to a scan image/slice are usedherein to refer to the relative thickness of the tissue represented inthe slice, which can vary depending on the scanner detector. It shouldbe appreciated that a thin slice has a smaller thickness than a thickslice. In accordance with most 3D medical imaging modalities (e.g., CT,MRI, PET, etc.), the native resolution of thin scan images (e.g.,obtained with thin detectors) is higher than the native resolution ofthicker scan images (e.g., obtained with thicker detectors). In thisregard, a thick scan image paired with corresponding thin scan image isassumed to have a lower resolution relative to the thin scan image. Forexample, the nominal slice thickness in CT is defined as the full widthat half maximum (FWHM) of the sensitivity profile, in the center of thescan field; its value can be selected by the operator according to theclinical requirement and generally lies in the range between 1millimeter (mm) and 10 mm. In general, the larger the slice thickness,the greater the low contrast resolution in the image, while the smallerthe slice thickness, the greater the spatial resolution. If the slicethickness is large, the images can be affected by artifacts due topartial volume effects, while if the slice thickness is small, theimages may be significantly affected by noise. The terms scanimage/slice are used herein to refer to the relative thickness of thetissue represented in the slice, which can vary depending on the scannerdetector. The terms “low-resolution” and “high-resolution” as usedherein refer to the relative resolution or MTF of two images, whereinlow-resolution image is interpreted as having a lower resolutionrelative to the high-resolution image.

One or more embodiments are now described with reference to thedrawings, wherein like referenced numerals are used to refer to likeelements throughout. In the following description, for purposes ofexplanation, numerous specific details are set forth in order to providea more thorough understanding of the one or more embodiments. It isevident, however, in various cases, that the one or more embodiments canbe practiced without these specific details.

Turning now to the drawings, FIG. 1 illustrates a block diagram of anexample, non-limiting computing system 100 that facilitates enhancingthe quality of 3D anatomy scan images using deep learning in accordancewith one or more embodiments of the disclosed subject matter.Embodiments of systems described herein can include one or moremachine-executable components embodied within one or more machines(e.g., embodied in one or more computer-readable storage mediaassociated with one or more machines). Such components, when executed bythe one or more machines (e.g., processors, computers, computingdevices, virtual machines, etc.) can cause the one or more machines toperform the operations described.

In this regard, computing system 100 can be and/or include variouscomputer executable components. In the embodiment shown, these computerexecutable components include a reception component 102, a trainingcomponent 104, a training image generation component 108, an enhancementcomponent 110, a denoising component 112 and a display component 118.These computer/machine executable components (and other describedherein) can be stored in memory associated with the one or moremachines. The memory can further be operatively coupled to at least oneprocessor, such that the components can be executed by the at least oneprocessor to perform the operations described. For example, in someembodiments, these computer/machine executable components can be storedin memory 116 of the computing system 100 which can be coupled toprocessing unit 114 for execution thereof. Examples of said and memoryand processor as well as other suitable computer or computing-basedelements, can be found with reference to FIG. 13 , and can be used inconnection with implementing one or more of the systems or componentsshown and described in connection with FIG. 1 or other figures disclosedherein.

Computing system 100 also includes an enhancement model 106 that can beor include a computer executable component. The enhancement model 106corresponds to a supervised machine learning model adapted to generateenhanced versions of input scan images by deblurring and increasing theMTF of the input scan images while maintaining their noisecharacteristics and inter-tissue contrast characteristics. The trainingcomponent 104 provides for training and developing the enhancement model106 and the enhancement component 110 applies the trained model to newscan images (e.g., scan image 124) to generate enhanced scan images inthe field (e.g., enhanced scan image 126). In the embodiment shown,subscript 1 is used to indicate the enhancement model 106 ₁ is in thetraining/development stage and subscript 2 is used to indicate theenhancement model 106 ₂ has completed training and executable by theenhancement component 110. The type of supervised machine learning modelused for the enhancement model 106 can vary. In one or more exemplaryembodiments, the enhancement model 106 can be or include a deep learningnetwork model, such as a convolutional neural network (CNN). However,other types of machine learning models are envisioned.

The reception component 102 can receive the image data used for modeltraining and/or model application/inferencing. In the embodiment shown,the training data includes low/high resolution training image pairs 122and the image data used for model application is represented by scanimage 124. It should be appreciated that although one scan image 124 isshown, the number of input scan image processed by the trainedenhancement model enhancement model 106 ₂ can include any number (e.g.,all scan images in a series processed sequentially or in parallel bymultiple instances of the trained model). The type of the scan imagesused for training and inferencing should be the same modality but canvary with respect to numerous other factors (e.g., orientation, regionof interest ROI, acquisition protocol, etc.). In various exemplaryembodiments, the scan images (e.g., the low/high resolution trainingimage pairs 122 and the scan image 124) are CT scan images. However, thedisclosed techniques can also be applied to MRI images, PET images, andother 3D medical imaging modality reconstructed images.

The low/high resolution training image pairs 122 include a plurality ofexemplary scan image pairs, wherein each pair includes a low-resolutionimage and a corresponding high-resolution image. In this regard, thelow-resolution image and the high-resolution image are consideredcorresponding or paired because they respectively depict the samestructure (e.g., an anatomical ROI, a phantom image, etc.) in the sameorientation yet with different resolutions or MTFs. In other words, thehigh-resolution image of each training image pair represents a higherresolution version of its corresponding low-resolution image. Asdescribed in greater detail below, the disclosed techniques employsupervised machine learning techniques to train the enhancement model106 ₁ to learn how to transform the low-resolution training images intotheir corresponding/paired high-resolution images.

The low/high resolution training image pairs 122 can begenerated/realized using various techniques in a direction where a thindetector and a thick detector configuration equivalent image can begenerated. In this regard, for most CT scanners (and other 3D medicalimaging systems), an image acquired with a thin detector and acorresponding one with thick detector equivalent is not realizable inall directions. However, a CT scanner detector will have similar PSFcharacteristic in all directions. This property is exploited to trainthe enhancement model 106 ₁ on image pairs generated in the directionwhere thin and corresponding thick scan images are realizable. Thetrained enhancement model 106 ₂ can then be used to deblur new scanimages captured/generated along other directions with similar pixelspacing, as long as the deblur factor of the model is less than PSF ofthe new direction. The trained enhancement model 106 ₂ can also be usedto enhance scan images captured/generated relative to the trainingdirection.

In this regard, the low/high resolution training image pairs 122 cangenerated/realized using various techniques. In some implementations,the low-resolution images can include thick scan slice images and thehigh-resolution images can include corresponding thin slice scan images.For example, the thin scan image of can correspond to a slice within thethickness of is paired thick scan image. In this regard, the thick andthin scan images can respectively depict the same anatomical region ofthe patient in the same orientation. With these embodiments, the thickscan images can be generated through retro-reconstruction from theircorresponding native thin scan images. In other embodiments, thecorresponding low- and high-resolution training image pairs can begenerated using focal-spot wobbling. Still in other embodiments, thelow-resolution images of the training image pairs can be generated usinga low-resolution scanner while the high-resolution images can begenerated using a different, high-resolution scanner. For example, thesame subject, structure, phantom image, or the like can be scanned witha low-resolution scanner in one direction to generate the low-resolutionimages and further scanned in the same direction using a high-resolutionscanner in the same direction to generate corresponding high-resolutionimages at the same positions. In any of the above described techniquesfor generating the low-resolution and corresponding high-resolution scanimages, in some implementations, the high-resolution scan image can begenerated with a comb filter and the corresponding low-resolution scanimage can be generated without a comb filter.

The number of training image pairs can vary. The training image pairsmay be generated from the same 3D anatomy scan and/or different 3Danatomy scans (e.g., for same and different patients). These images canbe generated from 3D anatomy scan data in only a direction where a thindetector and a thick detector configuration equivalent image can begenerated. The disclosed techniques are anatomy and acquisition protocolagnostic. In this regard, the 3D anatomy scan or scans used to generatethe training image pairs can represent the same or different anatomicalregions of interest (ROI) and be captured with same or differentacquisition protocols. In some embodiments, all of the training imagepairs can be generated from 3D anatomy scans captured from the samescanner (or different instances of the same scanner). In otherembodiments, the training image pairs can be generated from 3D anatomyscans captured from different scanners (e.g., different types ofscanners, different models, etc.) using scanner detectors with same orsimilar PSF characteristics.

In the embodiment shown, the low/high resolution training image pairs122 are generated relative to the N-axis and the new scan image 124 towhich the trained enhancement model 106 is applied at the inferencingstage after training (e.g., by the enhancement component 110) isgenerated relative to the M-axis. The parameters N and M are merelyarbitrary and used to denote two different axes of a 3D coordinatevolume/system. In various embodiments, the 3D coordinate volume/systemcan be or correspond to a Cartesian coordinate system that employs threecoordinate axes, each perpendicular or orthogonal to the other two atthe origin. In mathematics, these axes are usually labeled as x, y andz. In medical imaging, these reference axes are often labeled as axial(or transverse), coronal and sagittal. For example, there are threestandard anatomic planes that are generally used to display data for CTscans: axial (or horizontal), coronal (cutting the body into slices thatgo from anterior to posterior or posterior to anterior), and sagittal(cutting the body into slices that go from right to left or left toright). However, 2D CT scan images can also be generated relative toother planes (e.g., oblique or even curved planes). As described herein,reference to scan images being generated relative to one axis of a 3Dcoordinate system refers to the scan images being generated at differentpoints along the direction of the one axis such that each scan image isoriented relative to the same anatomical plane. In various exampleimplementations described herein, the N-axis corresponds to the z-axis(or sagittal and/or coronal axis) and the M-axis corresponds to thex-axis (or axial axis). However, it should be appreciated that theM-axis can include any axis relative to 3D anatomy scan data in which animage acquired with a thin detector and corresponding one with thickdetector equivalent is realizable, and the N-axis can include any otherorientation, including an axis that is non-orthogonal to the M-axis.

In the embodiment shown, the reception component 102 can receive thatlow/high resolution training image pairs 122 for usage by the trainingcomponent 104 to train and develop the enhancement model 106 ₁.Additionally, or alternatively, the training image generation component108 can generate the low/high resolution training image pairs 122 fromreceived (or otherwise accessible) 3D anatomy scan data. With theseembodiments, the training image generation component 108 can generatethe low/high resolution training image pairs either throughretro-reconstruction from native thin and corresponding thick detectorrealization or focal-spot wobbling. In some embodiments, thehigh-resolution image of each (or in some implementations one or more)training image pair can be a denoised image. For example, thehigh-resolution scan image can be received by the reception component102 in a denoised form. Additionally, or alternatively, the denoisingcomponent 112 can perform one or more existing medical image denoisingprocess to transform the high-resolution scan images into denoisedversions. The denoising component 112 may also denoise new scan images(e.g., scan image 124) prior to application of the trained enhancementmodel 106 ₂ thereto to generate the enhanced scan images (e.g., enhancedscan image 126). Some example denoising processes that can be used todenoise the denoising component 112 to denoise the input scan images caninclude but are not limited to: wavelet methods, curvelet methods,ridgelet based methods, sparse representation methods, shape adaptivetransform methods, bilateral filtering, principal component analysis(PCA), and nonlocal means (NL-means), and nonlinear variational methods.

In various embodiments, the training component 104 trains theenhancement model 106 ₁ using supervised machine learning to learn andperform one or more transformations between the low- and -highresolution paired images of the low/high resolution training image pairs122, wherein the high-resolution paired images are used as the groundtruth example. In this regard, the training process involves trainingthe enhancement model 106 ₁ to transform the low-resolution image intoits corresponding high-resolution image while maintaining the noisecharacteristics and tissue contrast characteristics of both images.

As noted above, in some embodiments, the low-resolution training imagescan comprise thick scan slice images while the high-resolution scanimages can include corresponding thin slice scan images. The thicknessof the respective images in each pair can vary so long as long as thethin image is thinner than its paired thick image. In one exampleembodiment in which the training images comprise CT images, the thinscan image of each training image pair can have a thickness of 0.625 mmwhile the thick scan image can have a thickness of 1.25 mm. However, itshould be appreciated that these thicknesses are merely exemplary. Owingto their thickness variations, the point spread functions (PSF) used togenerate the thick and thin images in the respective pairs aredifferent. That is, the PSF of the thick (low-resolution) image is widerthan the PSF of the corresponding thin (high-resolution) image.

In various embodiments, the enhancement model 106 ₁ can employ a deeplearning neural network that can be trained (e.g., by the trainingcomponent 104) using supervised machine learning to learn and perform adeblurring transformation between the low-resolution images and theircorresponding high-resolution images based on their respective PSFcharacteristics. More particularly, the enhancement model 106 ₁ can betrained to deblur the low-resolution image to achieve the PSF of itscorresponding high-resolution mage. For example, the training component104 can train the enhancement model 106 ₁ to deblur the low-resolutionmage a deblur amount controlled by the PSF of its correspondinghigh-resolution image. However, in order to be able to apply the trainedenhancement model 106 ₂ to new scan images generated relative to adifferent axis (e.g., scan image 124), the deblur amount should be lessthan the PSF associated with the different axis. In other words, thedeblurring amount of PSF and/or slice sensitivity profile (SSP) removedfrom the low-resolution image by the convolution performed by theenhancement model 106 ₁ can be constrained to be less than the minimumPSF and/or SSP that can be achieved in the applied/inferencing direction(which is a function of the imaging system used to acquire and generatethe input scan images in the applied/inferencing direction). Forexample, assume the enhancement model 106 ₁ was trained to remove 5units of PSF from the low-resolution scan images to achieve the PSF oftheir corresponding high-resolution scan images. In this example, thePSF of the new direction scan image (e.g., scan image 124) must begreater than 5. This is a minimal condition that needs to be maintained.

In this regard, the training component 104 can employ supervised machinelearning to train the enhancement model 106 ₁ to learn and perform oneor more non-linear transformations between the training image pairs. Theone or more non-linear transformations can include transformations basedon (but not limited to) tissue specific denoising, artifact correction,and HU-dependent smoothing/sharpening. For example, the trainingcomponent 104 can train the deep learning model to deconvolve tissuefeatures, contrast features, spatial features and/or PSF features ofthese components (e.g., tissue/contrast/spatial) between the trainingimage pairs under one or more user defined constraints. The one or moreuser defined constraints can include at least one of, an intensitythreshold constraint, a mask constraint, a spatial constraint, orcontrast distribution constraint. The training component 104 can employvarious loss functions to preserve the one or more user definedconstraints. Some suitable loss functions can include but are notlimited to: a mean MAE loss function, a percentage loss function, aperceptual loss function, an adversarial loss function, and a pointspread characteristics constraining loss function.

FIG. 2 presents an example framework for training a deep learningnetwork (i.e., enhancement model 106 ₁) to enhance the quality of 3Danatomy scan images in accordance with one or more embodiments of thedisclosed subject matter. As shown in FIG. 2 , the supervised trainingprocess involves training the enhancement model 106 ₁ to learn how totransform low-resolution scan images 202 generated relative to one axis(e.g., the N-axis) into their corresponding high-resolution scan images204 generated relative to the same axis. The training/learning processinvolves training the network to learn non-linear convolutiontransformations between the training image pairs. Once the network istrained and tuned, the network can be used to transform a scan imageinto a reconstructed higher resolution (or higher MTF) image thatresembles its corresponding thin scan image. The new scan image can havea same or similar resolution as the low-resolution training imagesand/or a higher or lower resolution than the low-resolution trainingimages. The new scan image can also have a different orientationrelative to the training images (e.g., generated along a different axisor plane). The training method is separable along each dimension in 3D,so it can be either done in 1D separately or 2D or 3D. If the PSF in thedirection of learning is same as all directions of inferencing, then thefilter dimensions can be reduced only to the required dimension (e.g.,1D) leading to a much lighter network. For example, if the PSF in thedirection of learning is same as all directions of inferencing, then theenhancement model 106 ₁ can learn the PSF convolution between thetraining image pairs along the row pixels in 1D (e.g., the x-direction)and apply it the column pixels in 2D (e.g., the y-direction) andadditional pixels in 3D (e.g., the z-direction).

In one example implementation, the low-resolution scan images 202 caninclude low-resolution CT scan images in the sagittal and/or coronalorientation (i.e., generated relative to the z-axis). Theircorresponding high-resolution scan images 204 can includehigh-resolution CT scan images in the same sagittal and/or coronalorientations (i.e., generated relative to the z-axis). The low/highresolution training images can be generated using one or more of thetechniques described herein (e.g., thick/thin retro-reconstructions,focal spot wobbling, usage of separate high- and low-resolutionscanners, and the like).

In embodiments in which the training image pairs include thick/thin scanslice images, the slice thickness (ST) of the thick images and the thinimages as well as the slice pacing (SS) can vary. In some embodiments,the ST of the thin images can be about half the slice thickness of thethick images. The SS between the thick images and/or the thin images canbe a function of the display field of view (DFOV) of the desiredenhanced output image when the network is applied by the enhancementcomponent to scan images with alternate orientations relative to theorientation of the training images. The DFOV determines how much of thescan field of view is reconstructed into an image. The pixel size of areconstructed image can be calculated by dividing the DFOV by the matrixsize. The matrix (also referred to as the grid), is the 2D grid ofpixels used to compose images on a display monitor. The matrixdetermines the number of rows and columns.

In this example, the slice thickness (ST) of the thick scan images 202is 1.25 mm while the ST of the thin scan images is 0.625 mm. The slicespacing (SS) between the thin scan images 202 and their correspondingthin scan images 204 can be kept the same to maintain the same imagedimensions and grid size. For example, in one implementation, the SSbetween the thick images and the thin images can both be 0.625.

Additionally, or alternatively, the high-resolution scan images 204 inthe training image pairs can include up-sampled images with double (orincreased by another factor) the grid size of their correspondinglow-resolution scan images 202. This can be accomplished for example, bydecreasing the SS of the high-resolution, thin scan images by half(e.g., 0.5×) to perform an up-sampling of the input image duringdeconvolution to further increase the grid size and the correspondingresolution/sharpness of the output image, resulting in asuper-high-resolution output image. With these embodiments, theenhancement model 106 ₁ can include a downstream up-sampling filter thatincreases the grid size of the input scan image in association withgenerating the output scan image, resulting in a super-high-resolutionoutput image.

FIG. 3 presents an example enhancement model architecture 300 inaccordance with one or more embodiments of the disclosed subject matter.The architecture employed in this example resembles a CNN withup-sampling. In accordance with this example, the enhancement modelreceives a low/normal resolution scan image 302 with an M-axisorientation (e.g., axial) and outputs a high-resolution scan image 314in the M-axis orientation, wherein the model was trained on scan imageswith an N-axis orientation (e.g., sagittal and/or coronal). Thearchitecture 300 includes head convolution layers 304, residual denseblocks 306, concatenation layers 308, an up-sampling layer, and tailconvolution layers 312. The grey filled boxes indicate multiple (e.g.,two or more) boxes or layers which can respectively include multiple(e.g., two or more) filters. The up-sampling layer 310 can be adapted toincrease the grid size of the input image by a desired factor (e.g., 2×or another factor), to generate super high resolution output image. Theup-sampling layer 310 can be removed to maintain the same grid size orincluded to provide for super resolution output.

With reference again to FIG. 1 , once the enhancement model 106 ₂ hasbeen trained on scan images in the N-axis orientation, the enhancementcomponent 110 can apply the model to new scan images in a differentorientation (e.g., scan image 124 with an M-axis orientation) to outputcorresponding enhanced scan images in the same orientation (e.g.,enhanced scan image 126 with an M-axis orientation). The output imagescan be displayed on a monitor/display screen by the display component118 for review by a clinician (e.g., a radiologist), stored, or exportedfor further processing (e.g., using one or more image processingmodels). The output images (e.g., enhanced scan image 126) have a higherresolution/MTF relative to the input images (e.g., scan image 124) witha same or similar noise level and inter-tissue contrast appearance. Invarious embodiments, the enhancement model 106 ₂ increases theresolution/MTF of the output image between about 30% and 80% relative tothe input image with less than a 10% change in noise characteristics andinter-tissue contrast appearance. In embodiments in which up-sampling isnot performed by the enhancement model 106 ₂, the enhanced scan image126 and the input scan image 124 will have the same size (e.g., samegrid size). In embodiments in which up-sampling is performed by theenhancement model 106 ₂, the enhanced scan image 126 will have a largersize (e.g., larger grid size) relative to the input scan image 124.

The architecture of system 100 can vary. For example, one or morecomponents of the computing system 100 can be deployed on the samecomputing device (e.g., the scanner device/system used toacquire/capture and generate the input scan images). Additionally, oralternatively, one or more components of the computing system 100 can bedeployed at different communicatively coupled computing devices (e.g.,via one or more wired or wireless communication networks) in adistributed computing architecture.

FIG. 4 presents a flow diagram of an example process 400 for employingthe enhancement model 106 ₂ to enhance the quality of 3D anatomy scanimages in accordance with one or more embodiments of the disclosedsubject matter. In accordance with process 400, the enhancement model106 ₂ was trained to learn the transform between low/high resolutiontraining image pairs generated along a first direction/axis in 3D whilethe input scan image were generated relative to a second direction/axisin 3D. In some embodiments in which the enhancement model 106 ₂ reformatimproves resolution only in one direction (e.g., 1D), the enhancementcomponent 110 can apply the enhancement model 106 ₂ two times whiletransposing the input scan image to enhance the input image in 2D (e.g.,wherein all scan images are 2D images).

In this regard, in accordance with process 400, the enhancementcomponent 110 can first apply the enhancement model 106 ₂ to alow-resolution input scan image having a first orientation that isdifferent from the training direction. In this example, the firstorientation is arbitrarily denoted as (A×B), which can correspond to anytwo orthogonal axes of a 3D coordinate system (e.g., X×Y, axial, etc.).The output of the first pass includes a partially enhanced scan image inthe same orientation as it was input. In this regard, for the firstpass, the enhancement model 106 ₂ enhances the characteristics of theinput image along one dimension or axis, which in this case is theA-direction. In embodiments in which the model is adapted to performup-sampling, the output scan image will also be up-sampled in theapplied direction. For example, the up-sampling can increase the gridsize in the applied direction by double (or 2×), resulting in the outputimage being (2A×B). In this regard, the enhance scan image can have asame or larger size (grid size) relative to the input image.

At 402, the enhancement component 110 then transposes the scan data tochange is input orientation to (B×A). This process can correspond toflipping the orientation of a 2D image from portrait to landscape (orvice versa). The partially enhanced scan image which has been enhancedalong the A direction only is then input to the enhancement model 106 ₂a second time to enhance the characteristics along the orthogonaldirection/dimension, the B-direction. The output of the second passincludes a fully enhanced scan image in the (B×A) orientation. Again, ifthe model is adapted to perform up-sampling, the output scan image willalso be up-sampled in the applied direction, resulting in the outputimage being (2A×2B).

The image is considered fully enhanced because it has now been enhancedin both dimensions of the 2D image, A and B. At 404, the enhancementcomponent 110 can transpose the scan data again to return itsorientation to the same orientation of the original input image, that isthe (A×B) orientation. The final result of process 400 is a fullyenhanced scan image in the (A×B) orientation which may be up-sampled toprovide a super resolution image (e.g., 2A×2B).

FIG. 5 presents a phantom CT image with and without deep learning-basedenhancement in accordance with one or more embodiments of the disclosedsubject matter. Image 501 depicts the input phantom CT image withoutenhancement model 106 ₂ application and image 502 depicts the outputphantom CT image with enhancement model 106 ₂ application. The MTF ofthe respective images is measured in line pairs per centimeter (lp/cm).As can be seen by comparison of the input image 501 and the output image502, the MTF of the output image 502 is higher than the input image 501.The enhanced resolution of the output image 502 relative to the inputimage 501 can also be clearly discerned. For example, the second blockof lines in the output image 502 are clearly sharper and moredistinguishable than the second block of lines in the input image 501.Furthermore, the third block of lines in the input image 501 are blurredand cannot be distinguished while the third block of lines in the outputimage 502 are clearly sharper and more distinguishable.

FIG. 6 provides a graph 600 comparing MTF test results on phantom CTimages with and without deep learning-based enhancement in accordancewith one or more embodiments of the disclosed subject matter. Thehorizontal axis (e.g., the x-axis) plots the line pairs that can beresolved per cm (lp/cm). The input curve line corresponds to scan imagesat different slices thickness without enhancement while the output curveline corresponds to the same scan images after application of theenhancement model. As can be seen in graph 600, the MTF lp/cm values forthe output scan images with enhancement at different slice thicknessesare clearly improved relative to the input scan images withoutenhancement. For example, the enhanced scan images achieve the same orsimilar MTF as the unenhanced scan images yet at a greater thickness,especially in the range between 1.2 lp/cm and 13.2 lp/cm. Moreimportantly, the curves for the enhanced images relative to theunenhanced images have substantially the same shape, which demonstratesthat aside from the sharper resolution, the other characteristics of therespective enhanced and unenhanced images (e.g., noise characteristics,inter-tissue contrast characteristics) remain substantially the same(e.g., the enhanced images are not distorted relative to the unenhancedimages).

FIGS. 7-9 present clinical MTF CT images before and after application ofthe disclosed enhancement model 106 ₂ in accordance with one or moreembodiments of the disclosed subject matter. The CT images shown inFIGS. 7-9 are respectively oriented in the axial direction and generatedwith an enhancement model 106 ₂ using process 400, wherein theenhancement model was trained on images in the z-direction.

FIG. 7 depicts a CT scan image of coronary artery with a stentpre-enhancement (e.g., input image 701) and post-enhancement (e.g.,output image 702) using the trained enhancement model 106 ₂. The stentis indicated with a dashed white circle in both images. As can be seenby comparison of the respective input and output images, the stent inthe output image 702 (as well as other visual features) has asharper/enhanced quality relative to the stent in the input image 701.For example, the stent strands and other features are clearlydistinguishable (e.g., separable) in the output image 702 while they arenot in the input image 701. In addition, the output image 702 and theinput image 701 are otherwise the same in appearance (e.g., the enhancedimage is not a distorted version of the enhanced image). In particular,the noise characteristics and the inter-tissue contrast characteristicsof the respective images are the same.

FIG. 8 depicts another CT scan image pre-enhancement (e.g., input image801) and post-enhancement (e.g., output image 802) using the trainedenhancement model 106 ₂. As can be seen by comparison of the respectiveinput and output images, the visual features of the output image 802have a sharper/enhanced quality relative to the input image 801. Inaddition, the output image 802 and the input image 801 are otherwise thesame in appearance (e.g., the enhanced image is not a distorted versionof the enhanced image). In particular, the noise characteristics and theinter-tissue contrast characteristics of the respective images are thesame.

FIG. 9 depicts another CT scan image pre-enhancement (e.g., input image1001) and post-enhancement (e.g., output image 1002) using the trainedenhancement model 106 ₂. As can be seen again by comparison of therespective input and output images, the visual features of the outputimage 1002 have a sharper/enhanced quality relative to the input image1001. In addition, the output image 1002 and the input image 1001 areotherwise the same in appearance (e.g., the enhanced image is not adistorted version of the enhanced image). In particular, the noisecharacteristics and the inter-tissue contrast characteristics of therespective images are the same.

FIG. 10 presents a high-level flow diagram of an examplecomputer-implemented process 1000 for enhancing the quality of 3Danatomy scan images using deep learning. Repetitive description of likeelements employed in respective embodiments is omitted for sake ofbrevity.

In accordance with process 1000, at 1002 a system operatively coupled toa processor (e.g., system 100 or the like), trains (e.g., via trainingcomponent 104) a deep learning network (e.g., enhancement model 106 ₁)to enhance the quality of first scan images generated from first 3Danatomy scan data relative to a first axis of a 3D volume. At 1004, oncetrained, the system employs (e.g., via enhancement component 110) thedeep learning network (e.g., enhancement model 106 ₂) to enhance thequality of second scan images generated from the first 3D anatomy scandata or second 3D anatomy scan data relative to a second axis of the 3Dvolume. The enhanced scan images may be rendered via display, stored,exported to another system, and so on.

FIG. 11 presents a high-level flow diagram of another examplecomputer-implemented process 1100 for enhancing the quality of 3Danatomy scan images using deep learning. Repetitive description of likeelements employed in respective embodiments is omitted for sake ofbrevity.

In accordance with process 1100, at 1102 a system operatively coupled toa processor (e.g., system 100 or the like) receives (e.g., via receptioncomponent 102) a scan image generated from 3D anatomy scan data relativeto a first axis of a 3D volume (e.g., scan image 124). At 1104, thesystem applies (e.g., via enhancement component 110) an enhancementmodel (e.g., enhancement model 106 ₂) to the scan image to generate anenhanced scan image having a higher resolution relative to the scanimage (e.g., enhanced scan image 126), wherein the enhancement modelcomprises a deep learning neural network model trained on training imagepairs respectively comprising a low-resolution scan image and acorresponding high-resolution scan image respectively generated relativeto a second axis of the 3D volume. The enhanced scan images may berendered via display, stored, exported to another system, and so on.

Example Operating Environment

One or more embodiments can be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product can include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium can be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network can comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention can be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions can executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer can be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection can be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) can execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It can be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions can be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionscan also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions can also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams can represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks can occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks cansometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

In connection with FIG. 12 , the systems and processes described belowcan be embodied within hardware, such as a single integrated circuit(IC) chip, multiple ICs, an application specific integrated circuit(ASIC), or the like. Further, the order in which some or all of theprocess blocks appear in each process should not be deemed limiting.Rather, it should be understood that some of the process blocks can beexecuted in a variety of orders, not all of which can be explicitlyillustrated herein.

With reference to FIG. 12 , an example environment 1200 for implementingvarious aspects of the claimed subject matter includes a computer 1202.The computer 1202 includes a processing unit 1204, a system memory 1206,a codec 1235, and a system bus 1208. The system bus 1208 couples systemcomponents including, but not limited to, the system memory 1206 to theprocessing unit 1204. The processing unit 1204 can be any of variousavailable processors. Dual microprocessors and other multiprocessorarchitectures also can be employed as the processing unit 1204.

The system bus 1208 can be any of several types of bus structure(s)including the memory bus or memory controller, a peripheral bus orexternal bus, or a local bus using any variety of available busarchitectures including, but not limited to, Industrial StandardArchitecture (ISA), Micro-Channel Architecture (MSA), Extended ISA(EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Card Bus, Universal Serial Bus(USB), Advanced Graphics Port (AGP), Personal Computer Memory CardInternational Association bus (PCMCIA), Firewire (IEEE 1294), and SmallComputer Systems Interface (SCSI).

The system memory 1206 includes volatile memory 1210 and non-volatilememory 1212, which can employ one or more of the disclosed memoryarchitectures, in various embodiments. The basic input/output system(BIOS), containing the basic routines to transfer information betweenelements within the computer 1202, such as during start-up, is stored innon-volatile memory 1212. In addition, according to present innovations,codec 1235 can include at least one of an encoder or decoder, whereinthe at least one of an encoder or decoder can consist of hardware,software, or a combination of hardware and software. Although, codec1235 is depicted as a separate component, codec 1235 can be containedwithin non-volatile memory 1212. By way of illustration, and notlimitation, non-volatile memory 1212 can include read only memory (ROM),programmable ROM (PROM), electrically programmable ROM (EPROM),electrically erasable programmable ROM (EEPROM), Flash memory, 3D Flashmemory, or resistive memory such as resistive random access memory(RRAM). Non-volatile memory 1212 can employ one or more of the disclosedmemory devices, in at least some embodiments. Moreover, non-volatilememory 1212 can be computer memory (e.g., physically integrated withcomputer 1202 or a mainboard thereof), or removable memory. Examples ofsuitable removable memory with which disclosed embodiments can beimplemented can include a secure digital (SD) card, a compact Flash (CF)card, a universal serial bus (USB) memory stick, or the like. Volatilememory 1210 includes random access memory (RAM), which acts as externalcache memory, and can also employ one or more disclosed memory devicesin various embodiments. By way of illustration and not limitation, RAMis available in many forms such as static RAM (SRAM), dynamic RAM(DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM),and enhanced SDRAM (ESDRAM) and so forth.

Computer 1202 can also include removable/non-removable,volatile/non-volatile computer storage medium. FIG. 12 illustrates, forexample, disk storage 1214. Disk storage 1214 includes, but is notlimited to, devices like a magnetic disk drive, solid state disk (SSD),flash memory card, or memory stick. In addition, disk storage 1214 caninclude storage medium separately or in combination with other storagemedium including, but not limited to, an optical disk drive such as acompact disk ROM device (CD-ROM), CD recordable drive (CD-R Drive), CDrewritable drive (CD-RW Drive) or a digital versatile disk ROM drive(DVD-ROM). To facilitate connection of the disk storage 1214 to thesystem bus 1208, a removable or non-removable interface is typicallyused, such as interface 1216. It is appreciated that disk storage 1214can store information related to a user. Such information might bestored at or provided to a server or to an application running on a userdevice. In one embodiment, the user can be notified (e.g., by way ofoutput device(s) 1236) of the types of information that are stored todisk storage 1214 or transmitted to the server or application. The usercan be provided the opportunity to opt-in or opt-out of having suchinformation collected or shared with the server or application (e.g., byway of input from input device(s) 1228).

It is to be appreciated that FIG. 12 describes software that acts as anintermediary between users and the basic computer resources described inthe suitable operating environment 1200. Such software includes anoperating system 1218. Operating system 1218, which can be stored ondisk storage 1214, acts to control and allocate resources of thecomputer 1202. Applications 1220 take advantage of the management ofresources by operating system 1218 through program modules 1224, andprogram data 1226, such as the boot/shutdown transaction table and thelike, stored either in system memory 1206 or on disk storage 1214. It isto be appreciated that the claimed subject matter can be implementedwith various operating systems or combinations of operating systems.

A user enters commands or information into the computer 1202 throughinput device(s) 1228. Input devices 1228 include, but are not limitedto, a pointing device such as a mouse, trackball, stylus, touch pad,keyboard, microphone, joystick, game pad, satellite dish, scanner, TVtuner card, digital camera, digital video camera, web camera, and thelike. These and other input devices connect to the processing unit 1204through the system bus 1208 via interface port(s) 1230. Interfaceport(s) 1230 include, for example, a serial port, a parallel port, agame port, and a universal serial bus (USB). Output device(s) 1236 usesome of the same type of ports as input device(s) 1228. Thus, forexample, a USB port can be used to provide input to computer 1202 and tooutput information from computer 1202 to an output device 1236. Outputadapter 1234 is provided to illustrate that there are some outputdevices 1236 like monitors, speakers, and printers, among other outputdevices 1236, which require special adapters. The output adapters 1234include, by way of illustration and not limitation, video and soundcards that provide a means of connection between the output device 1236and the system bus 1208. It should be noted that other devices orsystems of devices provide both input and output capabilities such asremote computer(s) 1238.

Computer 1202 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)1238. The remote computer(s) 1238 can be a personal computer, a server,a router, a network PC, a workstation, a microprocessor based appliance,a peer device, a smart phone, a tablet, or other network node, andtypically includes many of the elements described relative to computer1202. For purposes of brevity, only a memory storage device 1240 isillustrated with remote computer(s) 1238. Remote computer(s) 1238 islogically connected to computer 1202 through a network interface 1242and then connected via communication connection(s) 1244. Networkinterface 1242 encompasses wire or wireless communication networks suchas local-area networks (LAN) and wide-area networks (WAN) and cellularnetworks. LAN technologies include Fiber Distributed Data Interface(FDDI), Copper Distributed Data Interface (CDDI), Ethernet, Token Ringand the like. WAN technologies include, but are not limited to,point-to-point links, circuit switching networks like IntegratedServices Digital Networks (ISDN) and variations thereon, packetswitching networks, and Digital Subscriber Lines (DSL).

Communication connection(s) 1244 refers to the hardware/softwareemployed to connect the network interface 1242 to the bus 1208. Whilecommunication connection 1244 is shown for illustrative clarity insidecomputer 1202, it can also be external to computer 1202. Thehardware/software necessary for connection to the network interface 1242includes, for exemplary purposes only, internal and externaltechnologies such as, modems including regular telephone grade modems,cable modems and DSL modems, ISDN adapters, and wired and wirelessEthernet cards, hubs, and routers.

While the subject matter has been described above in the general contextof computer-executable instructions of a computer program product thatruns on a computer and/or computers, those skilled in the art willrecognize that this disclosure also can or can be implemented incombination with other program modules. Generally, program modulesinclude routines, programs, components, data structures, etc. thatperform particular tasks and/or implement particular abstract datatypes. Moreover, those skilled in the art will appreciate that theinventive computer-implemented methods can be practiced with othercomputer system configurations, including single-processor ormultiprocessor computer systems, mini-computing devices, mainframecomputers, as well as computers, hand-held computing devices (e.g., PDA,phone), microprocessor-based or programmable consumer or industrialelectronics, and the like. The illustrated aspects can also be practicedin distributed computing environments where tasks are performed byremote processing devices that are linked through a communicationsnetwork. However, some, if not all aspects of this disclosure can bepracticed on stand-alone computers. In a distributed computingenvironment, program modules can be located in both local and remotememory storage devices.

As used in this application, the terms “component,” “system,”“platform,” “interface,” and the like, can refer to and/or can include acomputer-related entity or an entity related to an operational machinewith one or more specific functionalities. The entities disclosed hereincan be either hardware, a combination of hardware and software,software, or software in execution. For example, a component can be, butis not limited to being, a process running on a processor, a processor,an object, an executable, a thread of execution, a program, and/or acomputer. By way of illustration, both an application running on aserver and the server can be a component. One or more components canreside within a process and/or thread of execution and a component canbe localized on one computer and/or distributed between two or morecomputers. In another example, respective components can execute fromvarious computer readable media having various data structures storedthereon. The components can communicate via local and/or remoteprocesses such as in accordance with a signal having one or more datapackets (e.g., data from one component interacting with anothercomponent in a local system, distributed system, and/or across a networksuch as the Internet with other systems via the signal). As anotherexample, a component can be an apparatus with specific functionalityprovided by mechanical parts operated by electric or electroniccircuitry, which is operated by a software or firmware applicationexecuted by a processor. In such a case, the processor can be internalor external to the apparatus and can execute at least a part of thesoftware or firmware application. As yet another example, a componentcan be an apparatus that provides specific functionality throughelectronic components without mechanical parts, wherein the electroniccomponents can include a processor or other means to execute software orfirmware that confers at least in part the functionality of theelectronic components. In an aspect, a component can emulate anelectronic component via a virtual machine, e.g., within a cloudcomputing system.

In addition, the term “or” is intended to mean an inclusive “or” ratherthan an exclusive “or.” That is, unless specified otherwise, or clearfrom context, “X employs A or B” is intended to mean any of the naturalinclusive permutations. That is, if X employs A; X employs B; or Xemploys both A and B, then “X employs A or B” is satisfied under any ofthe foregoing instances. Moreover, articles “a” and “an” as used in thesubject specification and annexed drawings should generally be construedto mean “one or more” unless specified otherwise or clear from contextto be directed to a singular form. As used herein, the terms “example”and/or “exemplary” are utilized to mean serving as an example, instance,or illustration and are intended to be non-limiting. For the avoidanceof doubt, the subject matter disclosed herein is not limited by suchexamples. In addition, any aspect or design described herein as an“example” and/or “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs, nor is it meantto preclude equivalent exemplary structures and techniques known tothose of ordinary skill in the art.

As it is employed in the subject specification, the term “processor” canrefer to substantially any computing processing unit or devicecomprising, but not limited to, single-core processors;single-processors with software multithread execution capability;multi-core processors; multi-core processors with software multithreadexecution capability; multi-core processors with hardware multithreadtechnology; parallel platforms; and parallel platforms with distributedshared memory. Additionally, a processor can refer to an integratedcircuit, an application specific integrated circuit (ASIC), a digitalsignal processor (DSP), a field programmable gate array (FPGA), aprogrammable logic controller (PLC), a complex programmable logic device(CPLD), a discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. Further, processors can exploit nano-scalearchitectures such as, but not limited to, molecular and quantum-dotbased transistors, switches and gates, in order to optimize space usageor enhance performance of user equipment. A processor can also beimplemented as a combination of computing processing units. In thisdisclosure, terms such as “store,” “storage,” “data store,” datastorage,” “database,” and substantially any other information storagecomponent relevant to operation and functionality of a component areutilized to refer to “memory components,” entities embodied in a“memory,” or components comprising a memory. It is to be appreciatedthat memory and/or memory components described herein can be eithervolatile memory or nonvolatile memory, or can include both volatile andnonvolatile memory. By way of illustration, and not limitation,nonvolatile memory can include read only memory (ROM), programmable ROM(PROM), electrically programmable ROM (EPROM), electrically erasable ROM(EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g.,ferroelectric RAM (FeRAM). Volatile memory can include RAM, which canact as external cache memory, for example. By way of illustration andnot limitation, RAM is available in many forms such as synchronous RAM(SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rateSDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM),direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), andRambus dynamic RAM (RDRAM). Additionally, the disclosed memorycomponents of systems or computer-implemented methods herein areintended to include, without being limited to including, these and anyother suitable types of memory.

What has been described above include mere examples of systems andcomputer-implemented methods. It is, of course, not possible to describeevery conceivable combination of components or computer-implementedmethods for purposes of describing this disclosure, but one of ordinaryskill in the art can recognize that many further combinations andpermutations of this disclosure are possible. Furthermore, to the extentthat the terms “includes,” “has,” “possesses,” and the like are used inthe detailed description, claims, appendices and drawings such terms areintended to be inclusive in a manner similar to the term “comprising” as“comprising” is interpreted when employed as a transitional word in aclaim. The descriptions of the various embodiments have been presentedfor purposes of illustration, but are not intended to be exhaustive orlimited to the embodiments disclosed. Many modifications and variationscan be apparent to those of ordinary skill in the art without departingfrom the scope and spirit of the described embodiments. The terminologyused herein was chosen to best explain the principles of theembodiments, the practical application or technical improvement overtechnologies found in the marketplace, or to enable others of ordinaryskill in the art to understand the embodiments disclosed herein.

What is claimed is:
 1. A system, comprising: a memory that storescomputer executable components; and a processor that executes thecomputer executable components stored in the memory, wherein thecomputer executable components comprise: a reception component thatreceives a scan image generated from three-dimensional scan datarelative to a first axis of a three-dimensional volume; and anenhancement component that applies an enhancement model to the scanimage to generate an enhanced scan image having a higher resolutionrelative to the scan image, wherein the enhancement model comprises adeep learning neural network model trained on training image pairsrespectively comprising a low-resolution scan image and a correspondinghigh-resolution scan image respectively generated relative to a secondaxis of the three-dimensional volume.
 2. The system of claim 1, whereinthe low-resolution scan image comprises a thick slice scan image and thecorresponding high-resolution scan image comprises a corresponding thinslice scan image generated using retro-reconstruction.
 3. The system ofclaim 1, wherein the low-resolution scan image and the correspondinghigh-resolution scan image were generated using focal spot wobbling. 4.The system of claim 1, wherein the low-resolution scan image wasgenerated without a comb filter and the corresponding high-resolutionscan image was generated with the comb filter.
 5. The system of claim 1,wherein the low-resolution scan image comprises a first scan imagegenerated using a low-resolution scanner and the correspondinghigh-resolution scan image comprises a corresponding second scan imagegenerated using a high-resolution scanner.
 6. The system of claim 1,wherein computer executable components further comprise: a trainingcomponent that employs supervised machine learning to train theenhancement model to learn and perform a deblurring transformationbetween the training image pairs under a deblur constraint based on oneor more point spread function characteristics associated with the firstaxis.
 7. The system of claim 1, wherein the enhanced scan imageresolution exceeds the maximum resolution of the imaging system on whichthe image was scanned along any axis of the three-dimensional imagevolume.
 8. The system of claim 1, wherein the enhanced scan image canhave a same size or a larger size in comparison to the input image. 9.The system of claim 1, wherein the enhancement model is separable inone-dimension, two-dimensions and three-dimensions.
 10. The system ofclaim 1, wherein computer executable components further comprise: atraining component that employs supervised machine learning to train theenhancement model to deconvolve point spread function using constraintsdriven by tissue features, contrast features and spatial featuresbetween the training image pairs under one or more defined constraints.11. The system of claim 10, wherein the one or more defined constraintsinclude at least one of, an intensity threshold constraint, a maskconstraint, a spatial constraint, or contrast distribution constraint.12. The system of claim 11, wherein the training component employs oneor more loss functions to preserve the one or more defined constraints,the one or more loss functions can be a mean absolute error lossfunction, a percentage loss function, a perceptual loss function, anadversarial loss function, and a point spread characteristicsconstraining loss function.
 13. The system of claim 1, wherein the scanimage is selected from the group consisting of: a computed tomographyimage, a multi-energy computed tomography image, and a magneticresonance image.
 14. A method comprising: receiving, by a systemoperatively coupled to a processor, receives a scan image generated fromthree-dimensional scan data relative to a first axis of athree-dimensional volume; and applying, by the system, an enhancementmodel to the scan image to generate an enhanced scan image having ahigher resolution relative to the scan image, wherein the enhancementmodel comprises a deep learning neural network model trained on trainingimage pairs respectively comprising a low-resolution scan image and acorresponding high-resolution scan image respectively generated relativeto a second axis of the three-dimensional volume.
 15. The method ofclaim 14, wherein the low-resolution scan image comprises a thick slicescan image and the corresponding high-resolution scan image comprises acorresponding thin slice scan image generated usingretro-reconstruction.
 16. The method of claim 14, wherein thelow-resolution scan image and the corresponding high-resolution scanimage were generated using focal spot wobbling or via scanning the samestructure using different scanners respectively comprising alow-resolution scanner and a high-resolution scanner.
 17. The method ofclaim 14, further comprising: employing, by the system, supervisedmachine learning to train the enhancement model to learn and perform adeblurring transformation between the training image pairs under adeblur constraint based on one or more point spread functioncharacteristics associated with the first axis.
 18. The method of claim14, further comprising: training, by the system, the enhancement modelto deconvolve point spread function using constraints driven by tissuefeatures, contrast features and spatial features between the trainingimage pairs under one or more defined constraints and one or more lossfunctions to preserve the one or more defined constraints, the one ormore loss functions can be a mean absolute error loss function, apercentage loss function, a perceptual loss function, an adversarialloss function, and a point spread characteristics constraining lossfunction.
 19. A machine-readable storage medium, comprising executableinstructions that, when executed by a processor, facilitate performanceof operations, comprising: training a deep learning network to enhancethe quality of first scan images generated from first three-dimensionalscan data relative to a first axis of a three-dimensional volume; andemploying the deep learning network to enhance the quality of secondscan images generated from the first three-dimensional anatomy scan dataor second three-dimensional anatomy scan data relative to a second axisof the three-dimensional volume.
 20. The machine-readable storage mediumof claim 19, wherein the training comprises training the deep learningnetwork to learn one or more transformations between training imagepairs respectively comprising a low-resolution scan image and acorresponding high-resolution scan image respectively generated relativeto the first axis of the three-dimensional volume, and wherein the oneor more transformation comprise a deblurring transformation between thetraining image pairs under a deblur constraint based one or more pointspread function characteristics associated with the second axis.